6

|

2 Fundamentals of Information, Signal and System Theory

through rare characters, i.e. for small values of pj. This connection is justified by the

fact that rarely occurring characters of a character set usually have a special meaning

in a message and therefore also carry a large information content. If one relates this

fact to the German language, less frequently occurring letters have a higher informa-

tion content and consequently a greater significance in the decoding of a text message.

In fact, based on the occurrence probability of letters in an encrypted message, a de-

cryption of the message can be achieved under certain conditions based on the total

occurrence probability of the individual letters in the German language.

Data Set

An important result of statistical information theory is the consistent description of in-

formation transmission in terms of symbols. The information-bearing symbols consist

of one or more information units, the so-called digital bits² with a word width index W

of up to 10 bits per symbol, since larger word widths can usually no longer be clearly

distinguished on the receiving side due to noise. Since a symbol has a different num-

ber of data bits depending on the coding, a distinction is made between the data rate

and the symbol rate of a data stream.

Data Transmission Rate

Determining the data stream of a digital signal is of great importance, for example,

for dimensioning a radio connection for data transmission from sensor networks via

Bluetooth. If one first defines the symbol duration Ts as the transmission duration for

a symbol, the so-called symbol rate or baud rate index fs = 1/Ts can be derived by

forming the reciprocal value. The baud rate denotes the number of transmitted sym-

bols per second. Common baud rates are between 9600 and 11, 520 baud (1 baud =

1 Bd = 1 symbol per second).³

The data transmission rate C is therefore the product of the symbol rate and the

word width C = D ⋅W and is given in bits per second. A Bluetooth 3.0 connection the-

oretically achieves data rates of up to 25 Mbit/s, so at 8 bits per byte a maximum of

3.125 MByte of data can be transmitted in one second.⁴ Other data rates for compar-

ison are given in Table 2.1.

2 bit is the short form for binary digit. 1 bit is the information content contained in a choice of two

equally probable possibilities. The information content can be any real, non-negative value. These

smallest information units have only two distinguishable states – zero and one. The number of bits

is called dataset D. A symbol of D bits therefore has Z = 2^D distinguishable states. In transmission

technology, it is common to work with 4 to 6 bit/symbol, which leads to 2⁴ = 16 or 2⁶ = 64 different

states for a symbol.

3 After Jean-Maurice-Émile Baudot, who invented the Baudot code in 1874.

4 The bit is usually abbreviated with a lower case b and the byte with an upper case B.